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ABSTRACT 

Communication  of  binary  data  over  an  additive  white 
Gaussian  channel  by  means  of  coded  transmission  over  coherent 
phase-shift  keyed  modems  is  considered.   It  is  found  that 
for  high  rate  codes  and  coarse  receiver  output  quantization, 
the  minimum  value  of  E./N  required  for  reliable  communica- 
tion is  significantly  higher  than  the  Shannon  limit  of 
-1.6  dB.  for  the  raw  channel.   This  loss  of  efficiency  is 
investigated  quantitatively  as  a  function  of  code  rate  and 
coarseness  of  quantization,  and  shown  to  be  negligible  for 
low  rate  code  with  fine  quantization.   Similar  results  are 
presented  showing  the  minimum  E,/N  ratio  for  operation  in 
the  region  where  a  sequential  decoder  performs  finite 
average  computation.   In  connection  with  this,  tables  and 
curves  are  given  which  show  the  value  of  the  Pareto 
parameter  p  as  a  function  of  E./N  . 
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I.   INTRODUCTION 

A.   CAPACITY  OF  THE  ADDITIVE  WHITE  GAUSSIAN  NOISE  CHANNEL 

Shannon's  formula  [Ref.  1]  for  the  capacity  of  the  band- 
limited  additive  white  Gaussian  noise  (AWGN)  channel,  shown 
in  the  communication  scheme  of  Figure  1  is 


1  P 

c   =   2  los2  ^  +  N~W^      bits/baud 

o 


p 
or  C   =  W  logpd  +  nT"w")       bits/second 


where  W  is  the  bandwidth  of  channel,  P  is  the  average  trans- 
No 
mitted  power,  and  -p-  is  the  noise  power  spectral  density. 

As  W  approaches  infinity,  i.e.,  the  bandwidth  is 

unlimited, 


C   =   wi!l   W  l0go(l  +  tttt) 
W-x»      &2      N  W 

o 


Letting  W  =  r- 


n     _     Lim  1  -    /,  ,  Ph 

C   "   h*0   h  l082(1  +  O 

o 


Llm  1   l"d  ^  Fh/NQ) 
h+°  h  '       in  2 


By  L'Hospital  rule 


C   = 


T  .     N   +  Ph 
Lim    o 

h*0 


In  2 


C   = 


N   In  2 
o 


bits/second 


This  is  the  capacity  for  the  infinite  bandwidth  AWGN  channel 
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Figure  1.   System  block  diagram 


Let  R  denote  the  number  of  information  bits  to  be 

transmitted  per  unit  time  (i.e.,  bits  per  second).   Then, 

P 
the  energy  per  bit  is  E.  s  s  .   Since  C  is  the  maximum 

number  of  bits  per  second  which  can  be  transmitted  with 

arbitrarily  small  error  probability,  must  have  reliable 

communication  requires  that  R  be  less  than  C,  or  equivalent ly 


p 
R  =  4-    <  c  = 


E,  Nrt  In  2 

D  O 


E. 
or   x~     >   In  2 

No 


Eb 
or   W~     >      0.69  approximately 

o 


Expressing  the  ratio  in  decibels,  this  becomes 


r 


E, 


N 
o 


dB   >   -1"6 


This  is  called  the  "Shannon  limit." 

The  significance  of  this  limit  is  that  no  matter  what  is 
in  the  boxes  labeled  "modulator"  and  "demodulator"  in  Figure 
1,  the  probability  of  error  can  be  made  arbitrarily  small 
only  if  E./N  exceeds  -1.6  dB;  conversely,  if  E.  /N  does 
exceed  -1.6  dB,  then  there  exists  some  "modulator"  and 
"demodulator"  to  make  the  probability  of  error  as  small  as 
desired. 

B.   DISCRETIZING  THE  CHANNEL 

1.   Bit-by-bit  Antipodal  PSK 

One  modulation  for  binary  communication  over  the 

AWGN  channel  is  the  antipodal  PSK  system  shown  in  Figure  2. 

The  data  stream  is  a  sequence  of  ±l's  so  that  over  each 

baud  of  length  T,  the  transmitted  signal  s(t)  is  a  cosine  of 
frequency  co  and  phase  either  0  or  tt.   Note  that  the  energy 


expended  for  information  bit  is 


T 
/   Cs(t)]2  dt 
0 


2EX 


I      [y±  —  COS  03c]   dt 


=  E, 


where  the  simplifying  assumption  is  made  that  a  baud  contains 
an  integral  number  of  cycles  of  the  cosine.   The  demodulator 
in  this  system  is  a  correlation  detector  which  requires  an 
oscillator  synchronized  with  that  at  the  transmitter;  the 
system  is  thus  called  a  coherent  system.   The  integrator 
output  at  time  T  is  tested  against  the  threshold  0.   If 
Z(T)  >,  0,  the  receiver  decides  that  the  data  bit  is  +1, 
if  Z(T)  <  0,  the  decision  is  -1.   The  system  operates 
identically  over  each  baud,  independent  of  all  other  bauds, 
and  is  therefore  called  a  bit-by-bit  system. 
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Figure  2.   A  model  of  bit-by-bit  antipodal  PSK  system 


Letting  R  again  denote  the  data  rate,  the  baud  length 
must  satisfy  T  =  =■  . 

Let  the  power  spectral  density  function  of  white 

No 
noise  be  -tj-  ;  thus  its  autocorrelation  function  is 

R(t)   =  <*-1  (-£) 
N 


Consider  in  the  case  of  signal  input  alone, 


±  /2Eb 
y(t)   =   "  J ~y~     cos  03ct 


from  the  integrator  output  at  time  T, 

+ ?  /f    t 

Z_(T)   -   ~— TfT-^-   /   cos2  0)  t  dt 
b  1      0  c 

=   +/e.     ,     assuming  that 

[0,T]  contained  integral  number  of  cycles  of  the  cosine. 
In  the  case  of  noise  present  alone,  the  output  is 


T  pp 
Zn(T)   =   /  J  |  n(t)  cos  wct  dt 


So,  when  noise  and  signal  are  presented  in  the  system 


y(t)   =   s(t)  +  n(t) 


Z(T)   =   Z  (T)  +  Z_(T) 

o  II 


=      ±  /Eh      +   /  /|  n(t)  cos  wct  dt 


The  quantity  Z  (T)  is  a  zero  mean  Gaussian  random  variable 

2 
whose  variance,  o        ,  can  be  found  as  follows: 

n 


n 


E[(Zn(T)r] 


=      EC 


T 


/        ^  n(t )    cos   a)  t   dt ) 
0     *  1  C 


E[| 


'    T  w   T  -j 

/  n(t)    cos   co  t   dt      /     n(s)    cos   u  s   ds    ] 
t0  c  J  l0  c  J 


T    T 


=     =r  /   /   E  n(t)*n(s)      cos   co  t-cos   us   dt   ds 
1   n  n      I  J  c  c 


But 


R(t)   = 
2 


E[n(t)-n(t-x)] 


T   T 


n 


m   /    /   R(t-s)    cos    to  t'cos    co   s    dt    ds 

1  0    0  c  c 

2  T  T  N0 

m   /   /     -y  6(t-s)    cos   co  t'cos   co   s   dt   ds 

1  0    0 

2  rT  N0  2 

£  J      —  cos      coct   dt 


10 


The  conditional  probability  density  function  of  Z(T)  given 
the  data  bit  can  be  represented  by  Figure  3- 


Pr[Z(T)  |  data  =  -1] 


Pr[Z(T)  |  data  =  +1] 


Z(T) 


Figure  3.   Conditional  probability  density  function  of 
Z(T)  in  the  case  of  a  positive  input  of 
amplitude  1,  and  in  that  of  a  negative  input 
of  amplitude  -1.  :. 


Then,  the  probability  of  error,  Pe ,  of  the  output 
stream  after  the  threshold  device  can  be  found  by 

Pe  =  Pr[(Z(T)  £  0)  |  data  bit  =  -1]  -  Pr[data  bit  =  -1] 
+  Pr[(Z(T)  <  0)  |  data  bit  =  +1]  •  Pr'[data  bit  =  +1] 

Assuming  equally  likely  data,  t 


Pr[data  =  -1]  =  Pr[data  -  +1]  -  ± 
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By  symmetry 

Pr[Z(T)  >  0  |  data  bit  =  -1]  =  Pr[(Z(T)  <  0  |  data  bit  =  +1] 

thus 

Pe  =  Pr[(Z(T)  >0)  |  data  bit  =  -1] 

or  Pe  =  Pr[(Z(T)  < 0)  |  data  bit  =  +1] 

Since  this  conditional  probability  density  function  is 

2   No 
Gaussian  with  zero  mean  and  variance  o     -  -^- 


J2v     o       0 


Pe   =   1      1     /    e-(x/20)  dx 


i   ir     -(y  /2)  ,, 


fe*      0 


where  y  =  -• 


^2VV      ,  2 


=  I  -  ±     J  e"^'/2)  dy 

*■  J2ti      0 


& 


The  probability  of  error  is  shown  by  the  shaded  area 
of  the  standard  normal  distribution  function  in  Figure  4, 
where 


f(y)    .    ^_   e-(y2/2) 

/2tt 
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In  Figure  5  the  curve  of  probability  of  error  versus 
E,/N  in  decibel  is  shown,  along  with  the  Shannon  limit. 
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Figure  4.   Probability  of  error.   Pe  =  Q( 
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Figure  5.   Probability  of  error  as  a  function  of  SNR, 
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It  can  be  seen  that,  for  example,  to  obtain  Pe  of  10" 
with  this  scheme,  (E,/N  },R  must  be  9.6   dB.   On  the  other 
hand,  the  capacity  theorem  says  that  there  is  some  system 
with  essentially  zero  error  probability  whenever  E,/N 
exceed  -1.6  dB  [Ref.  2].   Thus,  there  is  a  potential  gain  of 
11.2  dB  over  the  antipodal  PSK  system. 

2.   Antipodal  PSK  with  Rate  1/2  Coding 

Part  of  the  potential  gain  indicated  in  the  previous 
section  can  be  realized  when  the  somewhat  more  complicated 
system  of  Figure  6  is  used.   This  system  still  employs  a 
coherent  antipodal  PSK  modulation/demodulation  scheme,  but 
it  differs  from  the  previous  system  (Figure  5)  in  two  impor- 
tant respects:  (1)  more  bits  are  transmitted  per  unit  time 
than  appear  as  data  bits  (in  this  case,  twice  as  many);  and 
(2)  the  bits  transmitted  at  any  time  depend  not  only  on  the 
present  data  bit,  but  also  on  some  past  data  bits  (thus  this 
is  not  a  bit-by-bit  system) . 
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Figure  6.   A  model  of  antipodal  PSK  with  rate  1/2  coding 
system. 
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Let  data  stream  input  be  1  bit  for  each  T  second  interval, 
then,  for  code  rate,  R  =  1/2,  (R  is  defined  to  be  the  number 
of  information  bits  per  transmitted  bit),  the  output  from 
encoder  will  be  2  bits  for  each  T  second. 

Let  E  be  the  energy  expended  per  transmitted  bit  then 

5 

E  =  (l/2)E,  .   Assume  that  the  use  of  the  channel  is  as  fol- 
s      '  b 

lows:   each  bit  is  transmitted  by  binary  antipodal  signaling. 
At  the  receiver,  the  best  guess  is  made  for  each  bit  and  +1 
or  -1  is  the  output,  accordingly.   Thus,  from  the  output  of 
the  encoder  to  the  input  of  the  decoder,  the  channel  can  be 
though  of  as  a  discrete  system  with  binary  input  and  binary 
output  acting  as  illustrated  graphically  below,  where  p  is 
the  error  probability  for  bit-by-bit  binary  antipodal  signal- 
ing.  According  to  the  solution  in  bit-by-bit  antipodal  PSK 

2ES  % 
found  previously,  p  =  Q(  ■ -r= — ) 


•>  From  encoder       N/      To  decoder 


This  discrete  channel,  which  is  called  the  binary 
symmetric  channel,  has  a  capacity,  C,  with  the  following 
significance:   if  the  code  rate  R  satisfied  R  <  C,  then  there 
exists  a  code  (i.e.,  an  encoder-decoder  pair)  such  that  the 
error  probability  can  be  made  as  small  as  desired. 
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The  capacity  of  the  binary  symmetric  channel  is 


given  by 


where 


C  =  1  -  H(p) 


bit/use  of  channel. 


H(p)  =  -p  log2p  -  (l-p)log2  (1-p) 


p  -  Q< 


!E 


N 


) 


Thus  for  reliable  communication  in  this  system,  it  is 
necessary  that 


R  =  ^  <  C  =  1  -  H 


1  -  H 


or 


N 

No 

V          ) 

dB 


Q( 


2E. 


N 


2-E./2   ^ 

-T5T-5 


>   1.8  dB. 


When  this  condition  is  satisfied,  there  is  some  choice  of 
encoder  and  decoder  to  make  Pe.  as  small  as  desired. 

Note  that  this  minimum  E./N  is  still  3.^  dB  above 
the  Shannon  limit.   This  difference  is  due  to  use  of  R  =  1/2 
rather  than  a  lower  rate,  and  making  hard  decisions  at  the 
integrator  output.   In  Chapter  II,  the  minimum  E,  /N  is 
determined  for  various  rates  and  output  quantizations,  and 
it  is  shown  that  for  low-rate  codes  and  fine  output 
quantization,  the  Shannon  limit  is  approached. 
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C.   CAPACITY  OF  DISCRETE  CHANNELS 

Consider  a  discrete  memoryless  channel  (DMC)  whose  input 
alphabet  X  consists  of  the  K  integers,  0,  1,  2,  ...,  K-l, 
and  whose  output  alphabet  Y  consists  of  the  J  integers, 
0,  1,  2,  ...,  J-l.   Using  integers  for  input  and  output 
letters  simplifies  the  notation  somewhat  in  what  follows  and 
also  reemphasizes  the  fact  that  the  name  given  to  input  and 
output  letters  are  of  no  concern  whatsoever. 

The  channel  is  specified  by  a  transition  probability 
assignment,  P(j|k),  given  for  0  =  j  =  J-l,  and  0  =  k  =  K-l, 
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by  definition,  P(j|k)  is  the  probability  of  receiving  integer 
j    given  that  integer  k  is  the  channel  input.   For  example, 
see  Figure  8. 

Since  the  channel  is  memoryless,  each  output  letter  in 
the  sequence  depended  only  on  the  corresponding  input,  and 
the  probability  of  an  output  sequence  Y  =  (y_ ,  y_,  ...  y  ) 
given  an  input  sequence  X  =  (x, ,  x?,  ...  x  )  is  given  by 

N 
PM(Y|X)  =  IT   P(y  |x,  ) 

Let  Q(k)  be  the  probability  of  using  integer  k,  then  the 
average  mutual  information  [Ref.  3]  between  the  input  and 
output  is 

K-l  J-l                     pr  .  i.  v 
I(X;Y)  =  I        I      Q(k)  •  P(j|k)  log2  ^r^12 

k=°  J=°  2   Q(k)P(j|i) 

1-0 

where,  the  probability  of  receiving  integer  j  is  written  as 

|Q(i)P(j|i)  P(J|k) 


Y    k 


Figure  8.   Transition  probability  for  K  =  2  input, 

and  J  output 
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The  capacity  of  C  of  a  discrete  memoryless  channel  (DMC) 
is  defined  as  the  largest  average  mutual  information  I(X;Y) 
that  can  be  transmitted  over  the  channel  in  one  use, 
maximized  over  all  input  probability  assignments. 

C  =      max        I    Q(k)  P(j|k)  log2  ^Ufy 
Q(0)  ...  Q(K-lX  k,J 

C  is  the  function  only  of  the  channel.   The  calculation 
of  C  involves  a  maximization  over  K  variable  with  both  ine- 
qualities constrains,  Q(k)  =  0  and  an  equality  constrains, 
I    Q(k)  =  1.   The  maximum  value  must  exist  since  the  function 
is  continuous  and  the  maximization  is  over  a  closed  bounded 
region  of  vector  space. 

The  channels  of  interest  here  all  have  two  inputs  and 
are  symmetric,  as  shown  in  Figure  9,  so  that  the  capacity 
formula  reduces  to 

c  -fY  CP(j|o)  iog2  £$f  +  PCJU)  ios2  E#Jf] 
PC3|0)   =  P(J-j-l|l)   -  Aj+1 

J  A± 

C  =  1.0  +  I      A.  logp  ■ 

where  A,,  i=l,2,...3;  is  the  conditional  probability  of  receiving 
output  letter  j  given  that  input  letter  k  is  transmitted.  All 
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Then 
C  = 


details  of  modulator,  channel  and  demodulator  are  compressed 
into  the  probabilities  {A.} 


P(j=0)  =  ~(A1  +  k    ) 
P(j=l)  =  |(A2  +  AJ_1) 


P(j=J-l)  =  i(Aj  +  A1) 


Figure  9.   Conditional  probabilities  P(j|k) 

« 

for  J  output  and  K  =  2  input. 
P(j|0)  =  P(J-J-l|l)  =  A,+1 

The  amplitude  quantizer  may  be  one  of  the  five  shown  in 
Figure  10,  denoted  J  =  2,3,4,8  and  l6[Ref.  4].  Considering 
for  J  =  2,  K  =  2 


J           rti 
C  =  1.0  +   [  Ai  log^ 


i=l 


A.+AT  ... 
l   J-i+1 


=  1.0  +  An  log0  A..  +  A0    log0  A 


2  "1 


'2  "2 


=  1.0  +  plog2  p  +  (1-p)  lQg2  (1-p) 


2E 


where  p  =  A. 


1-p  =  A2.    p   =  Q( 


N 


*) 


thus  C  =  1.0  -  H(p),  channel  capacity. 
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Fig.  10.   Amplitude  quantizers  (a)  J=2,  hard 
decision,  (b)  J=3,  binary-symmetric 
erasure,  (c)  J=4  with  double  null  zone, 
(d)  J=8,  (e),  J=10  with  a=  t  /4  . 
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D.   SEQUENTIAL  DECODING 

A  sequential  decoder  is  one  which  decodes  by  making 
tentative  hypothesis  as  it  goes  along  and  by  changing  these 
hypothesis,  when  subsequent  choices  indicate  an  earlier 
incorrect  hypothesis  [Ref.  3,  page  263]. 

From  Gallager  [Ref.  3],  a  property  of  sequential  de- 
coding systems  is  that  the  computation  C  required  to  decode 
a  digit  is  a  Pareto  random  variable;  that  is,  its  distribution, 
for  large  L  assures  the  form 

Pr[C  >  L]  =  KL~P  as  L  approaches  °° 


where  p  is  the  solution  of  R  =  Rp 


EL(p) 

P 


R  =  bits/channel  use 


P 


Eo(p)    =   Max    {-logp    I   [    I     Q(k)    P(j|k)1/1+P]1+P} 
Q(k)  ^j=0    k=0 

0    <    p    <    °° 


where  the  general  formula  of  E  (p)  is  shown  in  Appendix  A. 

The  computation  a  sequential  decoder  must  perform  to 

decode  an  information  digit  is  a  random  variable,  and  Rpis 

the  rate  above  which  the  p   moment  of  this  random  variable 

fails  to  exist. When  p  =  1,  the  symbol  R     is  customary 

comp 

[Ref.  51.   In  order  to  operate  with  finite  average 
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computation.  R  must  be  less  than  R    .   This  condition  can 
r       '  comp 

be  translated  into  a  minimum  value  of  E, /N  in  the  same  way 

bo 

that  the  condition  of  R  <  C  yield  a  minimum  value  of  E./N  . 

bo 

In  Chapter  III,  minimum  values  of  E./N  are  given  for  various 
rate  and  quantization  levels. 
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II.   CAPACITY  LOSSES  DUE  TO  CODING  AND  OUTPUT  QUANTIZATION 


From  page  19 ,  the  channel  capacity  is 


C  =  1.0  +   J  A.  log, 


Ai 


1=1         Vaj-i+i 


Curves  for  various  amplitude  quantizers  as  shown  in  Figure  10 

are  presented  in  Figure  11.   The  curves  are  plotted  for  the 

value  of  E,  /N  required  to  achieve  C  =  R  versus  R  for 
bo   ^ 

J  =  2,3,^,8  and  16,  where  K  =  2. 

Table  I  gives  the  value  of  E. /N  vs  R  for  various 
values  of  J. 
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III.   R     LOSSES  DUE  TO  CODING  AND  OUTPUT  QUANTIZATION 
comp _ 


Curves  of  E,  /N  versus  R  are  presented  in  Figure  12  for 
bo 

various  values  of  J  to  achieve  p  =  1,  or  R     =  R.  where 

'      comp     ' 

K  =  2. 

Table  II  gives  the  value  of  E./N  vs  R.   Here,  as  in 
Table  I,  R  is  treated  as  a  continuous  variable,  although 
in  practice  R  is  usually  the  reciprocal  of  integer  to 
simplify  coder  and  decoder  design. 
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IV.       PERFORMANCE   OF    p   VERSUS   E,  /N      FOR  R   =    R 

D         O P 


E    (p) 

By  solving  R  =  R  ■  ,  curves  of  p  vs  E,  /N 

J  °  p     p    '  bo 

for  various  values  of  R  are  presented  in  Figures  13,  14  and 
15  for  J  =  2,4  and  8  respectively  and  are  useful  for  design 
tradeoff. 

It  will  be  seen  that,  for  example,  in  Figure  15  at 
p  =  1  the  value  of  Efe/N  is  2.65  dB  for  R  =  1/2  corresponded 
to  Figure  12  at  R  =  1/2,  and  at  p  =  0  the  value  of  Eb/NQ  is 
-0.66  dB  for  R  =  1/4  that  is  the  same  with  the  value  of 
Efe/No  in  Figure  11  for  J  =  8,  R  =  1/4. 
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TABLE  I 
Table  of  (E./N  )dB  Required  to  Achieve  C  =  R 


R 

J=2 

J=3 

J=4 

J=8 

J=l6 

3/4 

3.01 

2.34 

2.17 

1.76 

1.66 

2/3 

2.51 

1.75 

1.56 

1.18 

1.09 

1/2 

1.77 

0.90 

0.65 

-.30 

0.22 

1/3 

1.21 

0.28 

-0.04 

-.37 

-.45 

1/4 

0.97 

0.02 

-.32 

-.66 

-.74 

1/8 

0.65 

-.32 

-.70 

-1.04 

-1.13 

1/16 

0.51 

-.47 

-.88 

-1.22 

-1.31 

1/32 

0.44 

-.55 

-.96 

-1.30 

-1.40 

1/64 

0.40 

-.58 

-1.00 

-1.35 

-1.44 

1/128 

0.39 

-.60 

-1.02 

-1.37 

-1.46 

TABLE  II 

- 

(Eb/No)dB 

Required  to 

Achieve  p 

=  1,  R 

'   comp 

=  R 

R 

J=2 

J=3 

J=4 

-  J=8 _■'■ 

J=16 

3/4 

5.71 

4.54 

4.37 

3.67 

3-52 

2/3 

5.26 

4.10 

3.85 

3.25'  " 

3.11 

1/2 

4.59 

3.46 

3.18 

2.64. 

2.51 

1/3 

4.10 

2.99 

2.70- 

2.20 

2.08 

1/4 

3.89 

2.80 

2.51 

2.02 

1.90 

1/8 

3.  62 

2.55 

2.25 

1.78   ■- 

1.67 

1/16 

3.50 

2.44 

2.14 

1.68. 

1.56 

1/32 

3.44 

2.39 

2.09 

I.63  . 

1.51 

1/64 

3.41 

2.36 

2.06 

1.60 

1.49 

1/128 

3.39 

2.35 

2.05 

1.59. 

1.48 
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V.   SUMMARY  AND  CONCLUSION 

The  objective  has  been  to  present  a  unified  treatment 
of  efficiency  losses  due  to  coding  and  quantization  in  a 
PSK  system  with  additive  white  noise.   Performance  curves 
and  tables  are  given  for  useful  design  tradeoffs. 

In  Chapter  I  the  basic  concept  of  the  Shannon  limit  with 
respect  to  antipodal  bit-by-bit  PSK  is  presented.   In 
Chapter  II,. Ill,  and  IV,  tables  and  curves  showing  expected 
system  performance  are  presented.   The  expected  performance 
figures  were  obtained  by  computer-aided  analysis. 

It  is  shown  that  in  the  limit  of  fine  output  quantization 
and  low-rate  coding  the  Shannon  limit  is  approached. 
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APPENDIX  A 

GENERAL  FORMULA   OF  E  (p)  AND  R 

o  y  comp 


From  Gallager  [Ref.  3]  define  for  a  discrete  memoryless 
channel  with  K  input  symbols  and  J  output  symbols, 

Eo(p)   =   M^X  {-  log,  Y  [  Y  Q(k)P(j|k)1/1+P]1+P} 
°         W  j=0    k=0 

E0(p) 

and  R     =  — 

P         P 


Let   1+p   =   r  ,    Q(k)max  =  Q(k)     ,   K  =  2 

E_(p)   =   -  log?(Q(k))r  -  log.  I    [P(j|0)1/r+P(j|l)1/r]r 

d   j  =  0 

=   -  log2(Q(k))r  -  log2{[P(0|0)1/r+P(0|l)1/r]r 

[P(l|0)1/r+P(l|l)1/r]r 


+  [P(J-l|0)1/r+P(J-l|l)1/r]r} 


EQ(p)   =   -  log2(QXk))r  -  log2{(A^/r+Aj/r)r+(A^/r+A^)r+ 

+(A^/r+A];/r)r} 


where  A.,  i  =  1,2,3s*  ••»«?   is  the  conditional  probability 
of  j  given  k  as  shown  in  the  following  figure  for  K=  2. 
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k 


0 

1 

i 

2 

AJ+1   ■    P(j|0) 

3 

=    P(J-j-l|l) 

J-2 

J-l 

E0(P) 


*><h    <"£*  "  **2  t  I ,<*i/r+4&+1>r]  -A-l 


i=l 


For  Q(k)  maximum  =  p- 


Rcomp  "  "2-°  ^2  (l>  "  lo*2  CI  <Ai/2>'AJ-l+l>2] 


1=1 


2.0    -    log2[CA^2+A^2)2    +    (A21/2+A^)2    + 


+  (a/2+aJ/2)2] 


R 


comp 


J  J      ,-.  vl/2n 

2.0   -   log9    [2      J   A,    +   2      J      \ai   J-i+l;         J 
1=1   1  1=1     ( 


1.0   -   log2    [1.0   +      I    (^j_±+l)1/2^ 


•    •    •  }\  •  (— 
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